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Overview 



With their open admissions, low cost, and convenient locations, community colleges have taken 
great strides in recent decades in providing access to college for millions of students with diverse 
ethnic and academic backgrounds. Now, however, community colleges must tackle an even more 
formidable challenge: how to help increasingly large numbers of academically underprepared stu- 
dents succeed in college. The developmental courses to which over half of entering students are di- 
rected often prove to be too great a hurdle, and the majority who hope to earn a certificate or a de- 
gree, or to transfer, drop out before reaching their goals. 

Learning communities are a popular strategy that community colleges nationwide have embraced in 
support of developmental students. In a learning community, a cohort of students takes two or more 
courses linked by integrated themes and assignments that are developed through ongoing faculty 
collaboration. Learning communities are intended to foster active and collaborative learning and to 
create stronger relationships among students and between students and faculty. These elements are 
theorized to increase students’ motivation and sense of belonging, which in turn drive their effort, 
learning, persistence, and, ultimately, success. 

While the number of learning community programs continues to grow, rigorous studies measuring 
their effectiveness are limited. To address this need for evidence, the Learning Communities demon- 
stration, launched in 2007, uses random assignment to test models of learning communities at six 
community colleges: Kingsborough Community College, Queensborough Community College, 
Hillsborough Community College, Merced College, Houston Community College System, and 
Community College of Baltimore County. Five models serve developmental students in their first 
semester, and the sixth model enrolls second-semester students. The study is designed to answer 
three sets of questions: 

1 . How can learning communities be designed to address the needs of academically underprepared 
students? 

2. What are the effects of learning communities on student achievement, as measured by test 
scores, credits earned, and grades? What are the effects of learning communities on students’ 
persistence in higher education? 

3. What do learning communities cost and how do these costs compare with the costs of standard 
college programs for students with low basic skills? 

Preliminary findings will be available in 2009. This working paper describes the study’s design, in- 
cluding a summary of the theoretical and empirical research relevant to learning communities, de- 
scriptions of the sites and their learning community models, the random assignment procedures, and 
plans for data analysis. 
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Introduction 



Community colleges play a unique and indispensable role in higher education, each year 
offering millions a chance at a college education without regard to their level of academic prepara- 
tion. Community colleges now serve close to half of all undergraduates and disproportionately 
high numbers of undergraduates of color. 1 With their open admissions, low cost, and convenient 
locations, community colleges enroll a heterogeneous mix of older or dislocated workers returning 
to school for retraining, welfare clients seeking short-term job training, and younger students in 
pursuit of an associate’s degree or seeking to transfer to a four-year institution. 

Having taken great strides in recent decades in providing access to a postsecondary edu- 
cation for so many, community colleges must now tackle an even more formidable challenge: 
how to help large numbers of academically underprepared students succeed in college. As enroll- 
ment steadily grows, an alarmingly high and steadily increasing percentage of students score too 
low on diagnostic tests to enroll in credit-bearing, college-level classes. In most community col- 
leges, over half of entering students are directed to remedial education in reading, writing, and 
mathematics. 2 To make matters worse, at most, only about 70 percent of these “developmental” 
students pass all their precollege reading and writing courses, and only about 30 percent pass all 
their developmental math courses, even after multiple attempts. 3 Discouraged and unable to afford 
being students for long, the majority of developmental students end up dropping out without earn- 
ing a certificate or a degree and without transferring to a four-year institution. 4 

Colleges are also facing growing pressure to pay attention to data on access, retention, 
and persistence and to show evidence of their progress — or lack thereof — in improving stu- 
dent success rates. This drive toward increased accountability in the postsecondary sector is 
nowhere more clearly stated than in the conclusions of the Commission on the Future of Higher 
Education (the Spellings Commission) in 2006. Charged with making recommendations that 
would position colleges and universities to educate and train American’s future workforce, the 
19-member commission focused on four areas: access, affordability, the standards of quality in 
instruction, and the accountability of institutions of higher learning to their constituencies (stu- 
dents, families, taxpayers, and other investors in higher education). For the first time in their 
history, community colleges are being asked to routinely track and report on their completion 

'in 1999-2000, one out of three of all undergraduate students ages 19 to 23 were enrolled in the nation’s 
1,100 community colleges. Students of color, particular Hispanics, are overrepresented in community colleges, 
compared with four-year institutions (Horn, Peter, and Rooney, 2002). 

2 Dougherty (1994, 2003); McCabe (2000); Roueche and Roueche (1999). 

Attewcll, Lavin, Domina, and Levey (2006). In the present paper, the terms “developmental’’ and “re- 
medial’’ are treated as synonyms. 

4 Adelman (2004). 
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rates and “learning outcomes of students.” The need for affordable, effective, and feasible inter- 
ventions has grown even more urgent as a result of this development. 

An extremely popular choice for colleges nationwide is the adaptation of a century-old 
college reform called “learning communities.” Modem learning communities in community 
colleges typically last one semester, enroll between 20 and 30 first-year students together in two 
or three linked courses in which curricula are integrated and are loosely organized under an 
overarching theme and in which such pedagogical practices as active and collaborative learning 
and cross-disciplinary instruction and experiences are encouraged. The “theory of change” un- 
derlying learning communities posits that students in learning communities become more en- 
gaged in learning and in college life because they are more likely to interact socially and intel- 
lectually with the other students in their courses, form stronger relationships with faculty, and 
make connections across disciplines and between their academic and personal experiences. This 
increased social integration and intellectual engagement strengthens the motivation to pursue 
educational goals. 

Community colleges, in their quest for reforms that work for struggling students, have 
embraced learning communities in increasing numbers over the past few decades. With a few 
important exceptions, however, studies of learning communities have failed to create the know- 
ledge base to support this trend. Most of the research on learning communities has been unable 
to establish causal links between the intervention and outcomes, especially academic outcomes. 
More specifically, with only one or two exceptions, evaluations of learning communities and 
most interventions designed to help developmental college students succeed have failed to use 
the “gold standard” of random assignment design. 

A Request for Proposals issued by the Institute of Education Sciences (IES) of the U.S. 
Department of Education, calling for a focused program of research with an emphasis on expe- 
rimental methods, has set a new course for evaluation of programs in higher education. IES 
awarded a five-year grant to fund the National Center for Postsecondary Research (NCPR), a 
coalition of research organizations including the Community College Research Center at 
Teachers College, Columbia University; MDRC; and the University of Virginia. As part of this 
grant, MDRC, with its NCPR partners, launched a multicollege demonstration of learning 
communities in 2006. The Learning Communities demonstration, which builds on a similar 
study done by MDRC of learning communities at Kingsborough Community College, 5 tests six 
different models of learning communities in six community colleges across the country. Five of 
the models serve developmental students, and the sixth is designed for second-semester students 



5 Scrivener et al. (2008). 
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who have declared an occupational major. The evaluation is designed to answer three sets of 
questions: 6 

1. How can learning communities be designed and operated to address the 
needs of academically underprepared students with low basic English and 
math skills? Do such learning communities offer a classroom and college ex- 
perience that is substantially different from traditional remedial education 
programs? 

2. What are the effects of learning communities on student achievement, as 
measured by standardized test scores in English and math, credits earned in 
developmental and regular college courses, grades, and other outcomes? 

What are the effects of learning communities on students’ persistence in 
higher education? 

3. What do learning communities cost, and how do their costs compare with the 
costs of standard college programs for students with low basic skills? 

As of the end of 2007, all six Learning Communities demonstration sites were in opera- 
tion, and random assignment of students to program and control groups was well under way, 
but the earliest findings on impacts are not expected for at least three years. 7 

The purpose of the present paper — the first in a series of planned publications resulting 
from this study — is to present the rationale for launching a major national demonstration of 
learning communities in community colleges and to describe the research design, goals, and 
methodology of the study. The remainder of the paper is organized in three sections. The next 
section provides a context for the study by reviewing the history of and the literature on learning 
communities. Then the paper describes the sites selected for the study and their learning com- 
munities, touching on progress to date in enrolling students in the study and efforts to assist the 
six colleges in operating their programs as robust learning communities, with a high degree of 
fidelity to their design. Finally, the concluding section of the paper describes the plans for con- 
ducting the demonstration’s implementation, impact, and cost studies. 



6 Results from the demonstration will be informed by comparative analyses of the effectiveness of learning 
communities and other remediation strategies in selected states and systems in which the demonstration is tak- 
ing place. The data collection and analysis will be carried out by NCPR partners and is not addressed in the 
present paper, which focuses exclusively on the experimental evaluation of learning communities. 

7 An interim report focusing on the implementation of the Learning Communities demonstration is ex- 
pected to be released in 2009, and a second report, including impact findings, is scheduled for 201 1. A how-to 
guide on designing and operating an effective learning communities program — based on the experiences of 
the six demonstration sites — will follow in late 2011. 
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The Case for Learning Communities: 
Review of the Literature 



Community colleges currently enroll 46 percent of all undergraduates and have a stu- 
dent population far more representative of American society than four-year college and univer- 
sity populations. 8 Nationally, community colleges enroll 47 percent of black undergraduates, 55 
percent of Hispanic undergraduates, 47 percent of Asian/Pacific Islander undergraduates, and 
57 percent of Native American undergraduates. 9 Community colleges also serve as a primary 
entry point into postsecondary education for adults who have no previous higher education, 
low-income individuals, and first-generation students. 10 

The considerable literature on community college persistence indicates that most com- 
munity college students who are taking advantage of this unparalleled access to higher educa- 
tion are not likely to earn a degree or a certificate. Bailey and Alfonso report that of all first-time 
college students who entered a community college in 1995, only 36 percent earned a certificate, 
an associate’s degree, or a bachelor’s degree within six years. 11 

High attrition rates are also the result of large and increasing numbers of students enter- 
ing postsecondary education underprepared for college -level coursework — the bulk of whom 
are enrolling in community colleges. 12 Greene and Foster found that approximately two-thirds 
of recent high school graduates enter college each year, yet many of these students are unpre- 
pared academically for college -level material. 13 Research has shown that students with weak 
academic preparation are less likely to enroll and succeed in postsecondary education. 14 



The Need for Developmental Education 

Community colleges have responded to student underpreparedness by offering deve- 
lopmental courses. In 2000, 42 percent of first-year students at two-year colleges enrolled in one 
or more developmental reading, writing, or mathematics course. 15 Yet the effectiveness of post- 
secondary developmental education has been the subject of an ongoing debate among educa- 



8 Cohen and Brawer (1991); Gardiner (1994). 

9 American Association of Community Colleges (2007). 
l0 Cohen and Brawer (1996); Laanan (1995); Green (2006). 

1 'Bailey and Alfonso (2005). 

l2 Grimes and David (1999); Bailey, Leinbach, and Jenkins (2005). 
l3 Greene and Foster (2003). 

14 Adelman (1999. 2004); Horn and Kojaku (2001). 
l5 Parsad and Lewis (2003). 
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tors, policymakers, and the public. 16 Proponents argue that developmental education is neces- 
sary because it expands educational opportunities for underprepared students; critics counter 
that college-level remediation should be discouraged because offering courses covering content 
and skills that should have been learned in high school is both inefficient and costly to the high- 
er education system. 17 

Rigorous research on the relationship between developmental education and student 
outcomes is rare, and the findings are mixed. The results of two recent studies of remedial edu- 
cation in two-year colleges in Florida and two- and four-year colleges in Texas suggest limited 
benefits. The Florida study, for example, found that math remediation has a modest positive 
effect on persistence into the second year of college, and the Texas study found evidence that 
math remediation may lead to slightly better grades when students take their first college -level 
math course. However, neither study found that remediation leads to increases in credit comple- 
tion or to higher levels of degree attainment. The Texas study also found that remediation has a 
minimal impact on labor market performance. 18 

Evidence that developmental education bolsters student success is provided by a study 
that examined the effects of English and math developmental education on 28,000 students in 
four-year institutions in Ohio. Results suggest that students in developmental education are 
more likely to persist in college than students with similar test scores and backgrounds who are 
not required to take the courses. They are also more likely to transfer to a higher-level college 
and to complete a bachelor’s degree. 19 



Approaches to Educating Developmental Students 

Community colleges have responded to their alarming and stubbornly persistent failure 
rates by developing a broad range of programs to offer additional support to developmental stu- 
dents, including orientation seminars, tutorial sessions, discipline-specific help, learning assis- 
tance centers, learning labs, supplemental instruction, learning communities, and individualized 
learning programs. 20 Levin and Koski identify the key ingredients of successful interventions for 
underprepared students in college. Interventions should foster motivation by building on the 
interests and goals of the students and providing institutional credit toward degrees or certifi- 
cates; offer substance by teaching skills within a substantive or real-world context, as opposed 
to a more abstract approach; encourage curiosity and inquiry-based learning; promote indepen- 



16 Levin and Calcagno (2008). 

lv Hoyt and Sorenson (2001); Bennet (1994); MacDonald (1998). 
18 Calcagno and Long (2008); Martorell and McFarlin (2007). 
19 Bettinger and Long (2005). 

20 Jarvi (1998); Bailey and Alfonso (2005). 
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dent thinking; and use multiple approaches to learning, such as collaboration, teamwork, tech- 
nology, and tutoring. 21 

Learning communities — a curricular model that links two or more classes together for 
a cohort of students — potentially include all these ingredients. Accordingly, they are currently 
one of the most popular interventions being tried to help remedial students. According to the 
results of the Second National Survey of First-Year Academic Practices (a survey conducted in 
2002 by the Policy Center on the First Year of College), 62 percent of the 966 responding re- 
search universities, baccalaureate colleges, and community colleges offer a learning community 
program. Among the 341 responding community colleges, 60 percent offer a learning commu- 
nity program. 22 



The Origins of Learning Communities 

Several scholars in the field attribute the underlying ideas and practices of learning 
communities to John Dewey and Alexander Meiklejohn, linking Dewey’s ideas about the im- 
portance of social interaction and democracy in learning processes with the community-building 
and democratic practices of Meiklejohn’ s Experimental College at the University of Wiscon- 
sin. 23 However, others in the field challenge the historical origins of the model and suggest that 
the ideas of Meiklejohn and Dewey should be revisited in light of the goals of present-day 
learning communities. In their article on reconsidering learning communities, Talburt and 
Boyles state that the “compatibility of the educational ideas of Alexander Meiklejohn and John 
Dewey are debatable at best” 24 and go on to question how relevant these philosophies are to the 
present state of learning communities. 

After the closing of Meiklejohn’s Experimental College in 1932, similar innovative and 
experimental approaches to education emerged in the 1960s. The most notable are the Universi- 
ty of California-Berkeley’s integrated curriculum program, established by a student of Meikle- 
john’s Experimental College; and the learning community program at San Jose State College, 
established by Merv Cadwallader, who later brought his ideas to The Evergreen State College. 25 

The learning community model was revitalized, and a number of programs were devel- 
oped nationwide, following the founding of the Washington Center for Improving the Quality 



2l Levin and Koski (1998). 

“Barefoot (2002). 

23 For further reading, see Fogarty and Dunlap with others (2003); Lenning and Ebbers (1999); Smith 
(2001); Zhao and Kuh (2004). 

24 Talburt and Boyles (2005), p. 214. 

“Smith (2001). 
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of Undergraduate Education at The Evergreen State College in 1985. 26 In its initial days as a 
resource center, the Washington Center supported colleges responding to the National Institute 
of Education’s recommendations, one of which was that all postsecondary institutions establish 
learning communities. 27 In need of a model that would foster a sense of community among their 
diverse student bodies while simultaneously supporting the needs of their unprepared learners, 
community colleges were among the first to heed this recommendation. 28 



Learning Communities in Community Colleges Today: Theory of 
Change and Core Dimensions 

At community colleges, learning communities consist at a minimum of a cohort of stu- 
dents enrolled together in two or three linked courses. Learning communities typically last one 
semester and are usually offered to freshman students. The cohort typically consists of a group 
of students with a common course requirement (for example, developmental English or a histo- 
ry class). 29 Sometimes two developmental courses, such as mathematics and English, are linked 
to each other, and sometimes a developmental course is linked with a regular, college -level 
course such as sociology or psychology. Increasingly, colleges include a “student success 
course” in the link — an increasingly popular strategy to help students learn study and time 
management skills and how to navigate in the college environment. 

All learning communities are intended to foster social integration and collaborative 
learning and to strengthen curricular coherence. 30 The theory of change for learning communi- 
ties in community colleges builds on the well-documented finding that the relationships that 
students form with faculty and other students enable and encourage students to persist and suc- 
ceed in their educational pursuits. Collaborative learning and other experiences offered by learn- 
ing communities enhance a sense of belonging, which, in turn, leads to an increase in student 
effort; it is this effort and engagement in learning processes that drives student knowledge ac- 
quisition and the development of academically relevant skills. 31 

In addition to improving knowledge acquisition, learning communities are theorized to 
facilitate cross-curricular connections, thereby deepening learning and promoting higher-order 
thinking skills. Curricular integration, initiated by linking courses, allows students to more easi- 



26 Fogarty and Dunlap with others (2003); Smith (2001). 

_7 Study Group on the Conditions of Excellence in American Higher Education (1984). 
28 Fogarty and Dunlap with others (2003). 

29 Jaffee (2007); Maher (2005). 

30 Fogarty and Dunlap with others (2003); Tinto (1997); Zhao and Kuh (2004). 

31 Tinto (1993, 1997). 




ly make connections across disciplines and topics and with their own personal experience. 32 
Figure 1 illustrates these relationships as a logic model. 

The theory of change implies that certain core features of learning communities need to 
be in place for these positive outcomes to be realized. Empirically, of course, learning commun- 
ities do vary in their robustness and the strength of these features. The most significant varia- 
tions occur in implementation of the following dimensions: curricular integration; pedagogical 
strategies that encourage active, collaborative, and meaningful learning; faculty and student re- 
lationships and interactions; and the availability and integration of supplemental services. 

Curricular Integration 

“Integration” refers to curricular or programmatic linkages organized around common 
themes in order to construct shared, relevant teaching and learning experiences. Examples of 
integration include aligned syllabi, joint homework assignments, and project-based and other 
learning experiences that encourage drawing on materials in all the linked courses. The level of 
integration in learning communities varies from very little to deep integration and tends to de- 
pend on how courses are linked: 

• Student cohorts in unmodified courses. Students from a larger lecture- 
based class meet separately as a cohort, or subgroup, of this class, to discuss 
or write about the lecture’s content. Though the content of the smaller class 
will match that of the larger lecture, the faculty from the two classes general- 
ly do not collaborate. 

• Linked or clustered courses. Students enroll together as a cohort in two or 
more classes with integrative assignments and a unifying theme. The class 
instructors who work within this structure collaborate to link the courses. 

• Team-taught or coordinated studies programs. Students’ and faculty’s 
course loads are fully integrated. Students meet as a group with all the pro- 
gram’s faculty or in smaller groups, in an attempt to create an interdiscipli- 
nary and integrative approach to learning. 33 



32 Tinto (1997). 

33 Lardner and others (2005). Price (2005) offers a similar typology of learning communities, defined by 
the level of integration. 
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The Learning Communities Demonstration 
Figure 1 

Learning Communities in Community Colleges: A Logic Model 
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Instructional Strategies 

Learning communities create an environment in which certain instructional strategies 
that promote active, student-driven teaching and learning are more possible than in the tradi- 
tional classroom. “Active learning” refers to the use of pedagogy that promotes critical thinking 
through experiential, collaborative, and contextualized experiences that feel relevant and mea- 
ningful to students. Examples of this include field trips; service learning projects; group work, 
discussion, and debate in the classroom; and assignments that “force” students to think and 
problem-solve rather than memorize material. 34 

Social Integration 

Learning communities are designed to promote closer ties among students and faculty. 
Faculty in learning communities have greater opportunity to work together, to share successful 
practices, and to communicate with each other about their students. Students have a chance to 
build networks, form study groups, support each other, and learn from students in other ethnic 
or socioeconomic groups. 

Supplemental Student Support Services 

Learning communities create opportunities to provide students with more knowledge of 
and access to campus services that enhance academic and social learning, such as tutoring and 
academic counseling, career guidance, and clubs. Many learning communities integrate some of 
these services directly into the classroom, by including a tutor or arranging visits from career 
guidance specialists. Others link with a student success course, in which there is an emphasis on 
taking advantage of available campus resources, such as support services. 

Whatever the specific model of learning communities adopted, the practice is wide- 
spread, and hopes are high that this strategy will make a difference in retention and completion 
rates. Yet little is known about the effectiveness of learning communities, especially at commu- 
nity colleges. 



The Effectiveness of Learning Communities: A Review of the 
Literature 

Functioning in at least 40 percent of the nation’s community colleges, learning com- 
munities are widespread and are continuing to grow in number. However, while the number of 
programs grows and the enthusiasm for learning communities builds, rigorous studies measur- 

34 Stefanou and Salisbury-Glennon (2002). 
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ing their effectiveness are limited. While there is an extensive body of empirical literature de- 
voted to learning communities, the majority of these studies focus on learning communities in 
four-year postsecondary institutions, lack a proper comparison strategy that would allow causal 
relationships to be explored, or focus exclusively on such outcomes as social integration rather 
than on academic outcomes. There are, however, at least two important exceptions to this pat- 
tern. Engstrom and Tinto used longitudinal survey data to investigate effects on student beha- 
vior and persistence for students in learning communities at 13 community colleges and 6 four- 
year institutions across the country. 35 In a study that uses a random assignment design, MDRC 
reports that, compared with students in regular programs, students in learning communities at 
Kingsborough Community College progressed more quickly through English courses that were 
required for graduation and were more likely to be enrolled in college three semesters later. 36 
Both of these groundbreaking studies are discussed below. 

Learning Communities and Student Interaction and Engagement 

The research presented in the literature devoted to learning communities is generally 
encouraging and provides some evidence that cohort membership is related to a more positive 
college experience, stronger connections among students, and increased interaction around aca- 
demic activity. 

In an evaluation of 40 learning communities at Kingsborough Community College, re- 
searchers randomly assigned about 1,500 students into learning communities or a control group 
(in which students registered in unlinked courses). The study finds that students who were 
enrolled in learning communities were more satisfied with their overall college experience, ex- 
perienced a stronger sense of belonging to the college community, and were more engaged in 
learning. 37 In their major study of learning communities in 13 community colleges, Engstrom 
and Tinto find that students in learning communities had more positive views of both their 
classmates and instructors, had stronger perceptions of the support and encouragement that they 
experienced on campus, and were more likely to feel that their coursework emphasized higher- 
order thinking skills. 

Students in learning communities, according to some studies, are more likely to spend 
time with other students not only socially but also in academic pursuits. 38 When social and aca- 
demic activities merge, as they do in successful learning communities, students are able to learn 
from their peers and simultaneously develop meaningful relationships with them. 39 For exam- 

35 Engstrom and Tinto (2007). 

36 Scrivener et al. (2008). 

37 Scrivener et al. (2008). 

38 Engstrom and Tinto (2007); Tinto and Goodsell (2003); Tinto (1997). 

39 Tinto (1997). 
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pie, Tinto and Goodsell reported on the experiences of first-year students in a Freshman Interest 
Groups (FIGs) at a research university. They found that the collaborative learning that took 
place in the FIGs allowed students to form friendships while simultaneously participating in 
academically fruitful activities. 40 

In his small, qualitative study of 13 graduate students participating in a three-semester 
learning community, Maher found that continuity of knowledge, a shared sense of history about 
classroom developments, and a sense of “peer responsibility” emerged in student discussions of 
their cohort membership during the second half of their 10 months together. 41 

Additional support for cohort membership comes from a study conducted by Etelapelto 
et al., in which highly involved cohort members viewed other members of the group as motiva- 
tors for maintaining good study habits and pursuing their academic goals. 42 A small handful of 
studies, summarized below, examines the association between participation in learning com- 
munities in community colleges and such academic outcomes as course completion, grades, and 
persistence in college. 

Learning Communities and Academic Achievement Outcomes 

Some studies offer evidence that learning communities improve critical and problem- 
solving skills — sometimes referred to as “higher-order” thinking. In their study of the relation- 
ship between first-year students’ participation in learning communities and their motivation and 
cognitive learning strategies, Stefanou and Salisbury-Glennon found that there was a significant 
change in students’ cognitive strategies, including critical thinking and rehearsal, after partici- 
pating in a learning community. 43 These results support the findings of Tinto ’s 1997 study, 
whose participants — members of a Coordinated Studies Program — spoke of the relationships 
between their participation in this program and their increased ability to explore and practice 
concepts that they learned in class. 44 

Most studies that have attempted to measure how learning communities affect learning 
and other academic outcomes either fail to detect those effects or suffer from serious methodo- 
logical flaws. 45 However, the two pivotal studies referenced earlier — the Kingsborough Col- 
lege study and Engstrom and Tinto’s longitudinal survey analysis — are the exceptions to this 
pattern. Both examined the impact of learning communities on academic progress. While the 



40 Tinto and Goodsell (2003). 

4 'Maher (2005). 

42 Etelapelto, Littleton, Lahti, and Wirtanen (2005). 

43 Stefanou and Salisbury-Glennon (2002). 

44 Tinto (1997). 

45 McPhail, McKusick, and Starr (2006); Zhao and Kuh (2004). 
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findings from these studies are somewhat less encouraging than the findings discussed above on 
the effects of learning communities on social integration and engagement, both studies report 
positive — albeit modest — impacts on academic outcomes. 

In their study of learning communities at Kingsborough Community College, research- 
ers report that students moved more quickly through English courses that were required for 
graduation. However, positive impacts on course completion and credits earned diminished af- 
ter the semester in which students were enrolled in the learning communities, and no impacts 
were found on degree attainment. Results on persistence are mixed: No difference was observed 
in the percentages of students in the program and control groups who enrolled in the next seme- 
ster or the semester thereafter, but students in learning communities were 5 percentage points 
more likely to be enrolled three semesters later. 46 Interestingly, Engstrom and Tinto also report a 
5 percentage point difference in persistence one year later between students who were enrolled 
in learning communities and the comparison group. 47 

More such rigorous studies of learning communities are badly needed, particularly of 
impacts on the outcomes that are most worrisome for community colleges, such as progress 
from developmental to college-level coursework and overall persistence for at-risk groups. It 
may not be an overstatement to say that the shortage of trustworthy evidence on the effective- 
ness of learning communities inhibits progress in reducing the overwhelming numbers of stu- 
dents who start in community colleges but fail to succeed. Methodological problems character- 
ize most of the existing research on this and other interventions for remedial students. 

The Kingsborough Community College study used random assignment to create a 
comparison group for students in learning communities. Random assignment provides the best 
evidence for efficacy because any differences in outcomes can be attributed to the experience 
offered by learning communities, rather than to preexisting differences in characteristics be- 
tween students who enroll in learning communities and students who do not. 48 The Learning 
Communities demonstration, which grew out of the Kingsborough Community College study, 
uses a random assignment design to evaluate learning communities at six colleges around the 
country and carries the potential of significantly improving the information that policymakers 
and practitioners need to meet the needs of developmental students. 



46 Scrivener et al. (2008) 

47 Engstrom and Tinto (2007). 

48 Myers and Dynarski (2003); Coalition for Evidence -Based Policy (2005); What Works Clearinghouse 
(2006). 
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The Demonstration Sites and Study Intake 



This section provides an overview of the Learning Communities demonstration, with 
descriptions of the six selected sites and their learning community models, the random assign- 
ment procedures used to enroll students in the study, and the technical assistance efforts in place 
to support robust learning communities that are implemented with the highest possible fidelity 
to their designs. 



Site Recruitment Goals 

Consistent with national enrollment trends, the six colleges in the Learning Communi- 
ties demonstration each serve large numbers of low-income and academically underprepared 
students. However, the learning communities that were developed by these colleges in response 
to the similar needs of their students vary along several dimensions, such as what types of 
courses are linked and the level of curricular integration between the courses. These differences 
were central to the site recruitment process, as the colleges were chosen to reflect the widest 
possible range of experiences among students currently enrolled in learning communities at 
community colleges across the country. 

All the sites had previous experience running learning communities. A central guiding 
protocol for selection was the four-tiered model proposed by Derek Price to classify learning 
communities along a continuum of most basic — “a cohort of students taking at least two 
courses together” — to most integrated — “a cohort of students taking at least two courses to- 
gether as part of a coordinated studies program in which faculty team-teach an integrated curri- 
cula .” 49 At the time the sites were selected, learning communities at Community College of Bal- 
timore Country (CCBC) in Baltimore, Maryland, and at Kingsborough Community College 
(KCC) in Brooklyn, New York, tended toward the most integrated end of the spectrum; learn- 
ing communities at Hillsborough Community College (HCC) in Tampa, Florida, and at Hou- 
ston Community College System (HCCS) in Houston, Texas, fell closer to the most basic end; 
and learning communities at Queensborough Community College (QCC) in Queens, New 
York, and at Merced College in Merced (Central Valley), California, lay in the middle. 

Beyond capturing the range of learning communities that are available to community 
college students, primary programmatic criteria for a college’s participation in the study also 
included a reasonably strong contrast between the experiences of students in learning communi- 
ties and those of students enrolled in standard classes. This contrast was assessed through ob- 



49 Price (2005), p. 19 
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servations of learning communities and conversations with faculty and students at each site. 
Another programmatic concern was that the learning communities in the evaluation at each site 
had a common core course in their links — usually developmental math or developmental Eng- 
lish — in order to ensure consistency across learning communities within the same college. 

A number of additional operational benchmarks guided recruitment and selection ef- 
forts. Random assignment requires a large sample size to successfully measure effects; each 
college demonstrated that it had a large pool of interested and eligible students and that it would 
be able to generate enough demand for learning communities to make random assignment poss- 
ible. In addition, each college had to be willing to host multiple site visits and be willing to 
make necessary modifications in the normal registration procedures to accommodate study in- 
take and random assignment. 

Sites were selected and began random assignment throughout 2007. KCC, QCC, and 
HCC were the first cohort of sites to enroll in the study. The first cohort began enrolling and 
randomly assigning students in spring 2007 for the fall semester. The second cohort of sites — 
Merced, HCCS, and CCBC — began random assignment in fall 2007 for the spring 2008 seme- 
ster. Random assignment will continue for an additional three semesters, through spring 2009 
for the first cohort and through fall 2009 for the second cohort. 



Description of the Colleges and Their Learning Communities 

Two learning communities in the demonstration have developmental math as their core 
course, and three have developmental English. The core course is linked with another develop- 
mental course (in reading, writing, or math), a college -level academic course, or a student suc- 
cess course. 50 A sixth model supports student work in two college-level courses with enrollment 
in an integrative seminar. Table 1 presents these models in detail, and Table 2 presents selected 
characteristics of the colleges in the demonstration. 

Queensborough Community College (QCC) 

QCC is a midsize, single-campus community college. Its learning communities target 
first-year students who assess into the lowest levels of developmental math. Transfer students 
who have less than a semester of credits and returning students who have failed one of these 
courses are also eligible for enrollment in the learning communities. In fall 2007, the learning 
communities linked developmental math with developmental English or college-level English 



50 Student success courses, offering between one and three credits, are an increasingly popular strategy that 
colleges use to help students learn study and time management skills, how to navigate in the college environ- 
ment, and how to build self-esteem and a sense of responsibility for their own education. 
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The Learning Communities Demonstration 
Table 1 



Selected Characteristics of the Learning Communities in the Learning Communities Demonstration, by College 





Basic Configuration 




Other Features 






College 


Core Course 


Linked with: 


Tutoring or 
Supplemental 
Instruction 


Extracurricular 
Activities 
(field trips, 
service learning, 
and so on) 


Themed 

Links 


Master 

Learner 


Career 

Guidance 


KCC 


Integrative Seminar 


Two courses required for an occupational major 
(Accounting, Business Administration, Mental 
Health and Human Services, Early Childhood 
Education, and Pre-Nursing/ Allied Health) 


X 


X 


X 




X 


QCC 


Developmental Math 


Fall 2007: Developmental English or college- 
level English Composition 
Spring 2008: College-level English 
Composition, Speech, Business, or Sociology 






X 






HCC 


Developmental Reading 


Student Success course 






X 






Merced 


Developmental English 


Developmental Reading, Developmental Math, 
Student Success course, or college -level course 


X 




X 






HCCS 


Developmental Math 


Student Success course 


X 


X 








CCBC 


Developmental English 
or Reading 


College-level Health, Psychology, Speech, 
History, Computer Information Systems, or 
Sociology 








X 






The Learning Communities Demonstration 
Table 2 

Selected Characteristics of the Colleges in the Learning Communities Demonstration 





KCC 
Brooklyn, 
New York 


QCC 
Queens, 
New York 


HCC 

Tampa, 

Florida 


Merced 

Merced, 

California 


HCCS 

Houston, 

Texas 


CCBC 

Baltimore, 

Maryland 


Degree of urbanization 


Urban 


Urban 


Urban 


Rural 


Urban 


Suburban 


Total enrollment (FTEs) a 


14,687 


13,150 


21,293 


10,116 


45,526 


19,446 


Enrollment by race/ethnicity (%) 


White non-Hispanic 


38.6 


24 


52.9 


35.6 


21.4 


56.3 


Black non-Hispanic 


28.9 


25.2 


18.7 


6 


25.8 


30.4 


Hispanic 


13.4 


20.6 


21 


39.2 


27.9 


2 


Asian/Pacific Islander 


11.1 


19.9 


3.9 


11.8 


11.3 


4.4 


American Indian/ Alaskan 


Native 


0.1 


0.3 


0.4 


1.1 


0.2 


0.4 


Nonresident alien 


7.9 


10 


2.1 


0.9 


9 


1.8 


Students receiving financial aid (%) 


73 


67 


57 


62 


56 


46 


First-time student retention rate b (%) 


Full time 


65 


69 


60 


64 


60 


59 


Part time 


40 


58 


45 


43 


47 


44 


Graduation rate c (%) 


24 


14 


26 


19 


11 


10 


Transfer-out rate (%) 


20 


23 


13 


9 


19 


13 



SOURCE: All data are from the Integrated Postsecondary Education Data System (IPEDS) (2006). 



NOTES: 

“Enrollment totals represent “full-time-equivalent” (FTE) students. 

b Retention rates measure the percentage of entering students who continue their studies the following fall. 

Graduation and transfer-out rates are calculated for full-time, first-time undergraduates who began their program in 2003. Graduation rates measure 
the percentage of entering students who complete their program in a certain time. Transfer-out rates measure the percentage of entering students who 
transfer to another institution within 150 percent of the normal time to program completion. 




composition; in spring 2008 and beyond, developmental math is linked with various college- 
level courses to better match students’ needs and interests. 

The links offered in spring 2008 have themes such as “Money Talks and Numbers 
Speak” (linking math and speech). Professors visit each other’s classes and meet before and 
throughout the semester to coordinate assignments. 

Houston Community College System (HCCS) 

HCCS is a very large, multicampus community college. As of spring 2008, Northline 
campus is the only one participating in the study. HCCS’s learning communities link develop- 
mental math and a student success course. They target first-time students who assess into the 
lowest levels of developmental math. The student success course is designed to prepare students 
for the demands of college courses, by emphasizing study skills and accessing campus re- 
sources. Math faculty collaborate with student success course instructors in order to maximize 
opportunities for students to make connections with the math curriculum. A tutor, who is either 
a staff member or another student, is assigned to each math class. 

Hillsborough Community College (HCC) 

HCC is a large, four-campus community college. Two of the campuses, Dale Mabry 
and Ybor, are currently participating in the study. HCC’s learning communities link develop- 
mental-level reading with a student success course and are targeted to first-time students who 
assess into developmental reading. The student success course is a credit-bearing class that fo- 
cuses on acclimation to college, study skills, and other tools to help students do well in their 
reading course. Faculty assign common work within links. 

Merced College 

Merced is a midsize, single-campus community college. Its learning communities link 
developmental-level English with either developmental reading, developmental math, or a stu- 
dent success course. The learning communities are designed for both new and returning stu- 
dents who assess into developmental English. Professors visit each other’s classes and coordi- 
nate assignments. Several of the links have Supplemental Instructors — trained peer instructors 
who facilitate voluntary group study sessions. 

Community College of Baltimore County (CCBC) 

CCBC is a large, multicampus community college. Two of the campuses, Essex and 
Catonsville, will be included in the study in spring 2008; a third, Dundalk, will join in fall 2008. 
Learning communities at CCBC link the highest level of developmental English or reading with 
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a college -level course, supplemented by a Master Learner component. The learning communi- 
ties target all students in need of one of the English or reading courses. The Master Learner is a 
faculty member (sometimes the developmental instructor) who sits in on the college -level 
course and conducts a weekly, one -hour, noncredit seminar on leaming-to-leam in the context 
of the college-level course. Links often feature joint themes and overlapping assignments. 

Kingsborough Community College (KCC) 

KCC is a large, single-campus community college. In 2003, MDRC launched an impact 
evaluation of KCC’s first-semester learning communities as part of the Opening Doors demon- 
stration. When KCC decided to develop learning communities with an occupational focus, 
MDRC approached the college again to participate in the current demonstration. This model is 
growing in popularity nationwide and is highly relevant to the problem of critical shortages of 
skilled workers in such occupations as nursing. 

KCC’s current learning communities target continuing or transfer students who have 
declared one of five occupational majors and are designed to strengthen the students’ industry 
knowledge and academic and study skills in order to support their work in occupation-oriented 
courses. The learning communities link an “integrative seminar” with two courses required for 
the major. The two content courses are three or four credits each and vary by link. The integra- 
tive seminar is a single-credit course that meets for two hours a week and is taught by a KCC 
faculty member from the given major. Representatives from employers visit the seminar, and 
students also visit workplaces in the local community. In addition, each seminar has an adjunct 
English faculty member to provide support and guidance for assignments from the content 
courses. Linked professors and seminar leaders meet regularly to work on course integration, 
faculty development, and program development. 



Study Intake and Random Assignment 

Study intake and random assignment, which began in spring 2007 for the first cohort of 
sites and in fall 2007 for the second cohort, will continue for a total of four semesters, to result 
in a sample at each college of approximately 1,000 students enrolled in the study. This sample 
size is judged to be sufficient to detect modest impacts. The study intake process, although 
roughly similar in each of the six sites, was adapted to fit the registration process at each cam- 
pus. 51 The following is a generic description of how students are enrolled in the study. 



51 Random assignment studies are rare in community colleges. The process of fitting study procedures — 
including informed consent, a questionnaire, and random assignment — into the normal enrollment and regis- 
tration process is complex and demanding. All six colleges, and particularly the advising staff, worked closely 
with MDRC staff to design and implement these intake procedures. 
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At the time they register for classes, students are screened for study eligibility, and they 
meet with a study representative from the college (typically an academic adviser or a registrar), 
who informs them about the learning communities program and the evaluation. The explanation 
includes a description of random assignment as a lottery-like process that is used to determine 
who can enroll in a learning community as well as information about their rights as study sub- 
jects. Before they go through random assignment, students who agree to participate in the study 
are asked to sign an informed consent agreement to indicate their understanding of their role in 
the evaluation and also to complete a Baseline Information form (BIF). The BIF is designed to 
capture information about the students’ demographic characteristics and educational goals, and 
it varies slightly across sites. 

Students are randomly assigned into the program group, which enrolls in a learning 
community, or the control group, which enrolls in the standard courses available to all students 
at the college. Generally, students are assigned using a 50:50 ratio. However, at some colleges, 
this ratio was changed to 60:40 during the first semester of random assignment, to ensure that 
the learning communities scheduled by the college were filled. To maximize contrast between 
the two research samples, students in the control group generally agree not to enroll in learning 
communities that involve developmental classes for at least two years after random assignment, 
but enrollment restrictions vary somewhat by site and are specified on the informed consent 
agreement. For example, students in the control group at HCCS are not allowed to enroll in any 
learning communities during the first semester, but they are allowed to enroll in nonmathemat- 
ics learning communities in subsequent semesters. 



Strengthening the Learning Communities Through Technical 
Assistance 

From its inception, the Learning Communities demonstration has been concerned with 
conducting a fair test of the learning communities model. A fair test of learning communities 
depends on strong program implementation and a distinct contrast from the classroom expe- 
rience outside the learning communities. Both the colleges and the researchers seek to ensure 
that each learning community model is implemented as intended by each site, that each program 
is as robust as possible, and that the contrast between the experience of students enrolled in the 
learning communities and students enrolled in the control group is as sharp as possible. To this 
end, MDRC provides technical assistance to each site in the demonstration. 

As of December 2007, three technical assistance consultants have been involved with 
the Learning Communities demonstration. Additional consultants may be added as sites en- 
counter new or different implementation challenges. Each of the technical assistance providers 
chosen to work with the sites has an extensive background in learning communities, with exper- 
tise in specific areas, such as curriculum integration and professional development. The consul- 
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tants have conducted site visits to most of the colleges to work with faculty and college admin- 
istrators on such topics as faculty collaboration and pedagogical practices in learning communi- 
ties. These visits will continue throughout the demonstration. 

A three-day summer institute in 2007 brought together teams from seven colleges, five 
of which are currently in the study. The institute focused on integration (social and academic), 
faculty collaboration, peer collaboration, active learning instructional strategies, administrative 
support, assessment of student work for evidence of integrated learning, and other topics that 
stimulated rich discussions of what good learning communities should look like. Follow-up 
phone calls and visits were conducted to learn how sites implemented these ideas and to offer 
support to college administrators and faculty. 
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The Research: Goals, Methodology, and Data 



This section describes the implementation study, the impact study, and the cost study. 
While the quantitative impact findings from the impact study are the primary focus of the re- 
search, findings from the implementation and cost studies contribute to a well-informed inter- 
pretation of the impact findings. 



The Implementation Study 

Implementation studies help researchers interpret and explain the impacts — or lack of 
impacts — of interventions. Implementation data can provide a glimpse into the “black box” of 
the intervention: in this case, the teaching, learning, and social interaction that goes on both in- 
side and outside the classrooms of learning communities. A guiding question for the implemen- 
tation study asks whether the learning communities as delivered are robust and incorporate the 
core dimensions of the model. Robust learning communities that adhere to those core dimen- 
sions should create an experience that is clearly distinct from that of students and instructors 
teaching and learning within standard, stand-alone developmental courses. 

Implementation data can also serve to measure “fidelity,” or the extent to which the in- 
tervention is implemented close to its original design over time, across learning communities, 
and across colleges. This is particularly important for the Learning Communities demonstration, 
as considerable variation in implementation is expected both across and within colleges. This 
happens primarily because, despite their popularity, learning communities can be challenging to 
implement even in their most basic form. For example, grouping students into cohorts and 
block-scheduling classes is an unfamiliar task for most community colleges. Designing and de- 
livering the more “high-end” learning communities — complete with a thematic focus, a fully 
integrated curriculum, and interactive instructional strategies — can be a daunting task in an 
institution where faculty rarely collaborate with each other on curricular matters. 

With these considerations in mind, the implementation study is designed to: 

1 . Document how the learning communities programs were initiated, designed, 
and operated at the participating colleges 

2. Identify factors that promoted or impeded the smooth implementation of the 
learning communities programs at each site and document variation from 
year to year 

3. Measure variation in implementation across learning communities within the 
same college and across the six colleges 
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4. Compare the classroom and college experiences of instructors and students in 
both the program group and the control group in order to determine the de- 
gree of contrast between the two experiences 

Measuring the Implementation of Learning Communities 

The six colleges in this demonstration have each developed a learning communities 
model that addresses their students’ specific needs, fits their institutional capacities, and reflects 
their own “theory of change.” But these models have all evolved from a common history, body 
of research, and set of beliefs about what learning communities in community college settings 
should look like. Most researchers and practitioners agree that a set of “core dimensions” — 
when taken together and implemented with a reasonable degree of fidelity — results in robust 
learning communities. These core dimensions constitute a framework to guide the implementa- 
tion study. They are shown in Table 3, along with examples of indicators of their robustness. 
(See Appendix A for a full list of indicators.) The extent to which the colleges in the demonstra- 
tion succeed in implementing learning communities with most or all of these dimensions is a 
measure of the fidelity of the implementation of the learning community model. 

Data collection for the implementation study will be geared toward measuring indica- 
tors of these core dimensions as well as the extent to which important site-specific features of 
learning communities (such as the integrative seminar at KCC or the Master Learner at CCBC) 
are implemented as the colleges intended. Some indicators will be measured using a rating sys- 
tem that will generate a numeric “fidelity score” for certain dimensions. This will allow for an 
overall ranking of the six colleges along a continuum of fidelity. For example, fidelity indices 
will be constructed to rate the degree to which curriculum is aligned or integrated across two or 
more courses in a given learning community. Researchers will use instruments to record ratings 
based on interview and focus group responses, classroom observations, and faculty survey res- 
ponses. 



Data Collection Activities 

The implementation study will rely on a variety of data collection strategies, a complete 
list of which is included in Appendix A. This section describes the three primary data sources 
for the implementation study: site visits, a faculty survey, and a review of existing documents 
and materials. 

Site Visits 

The implementation study will rely heavily on qualitative data gathered during research 
site visits to the colleges. Two site visits will be conducted at each college over the course of the 
demonstration, during the second semester following random assignment (spring 2008 for the 
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The Learning Communities Demonstration 
Table 3 

Core Dimensions of Learning Communities in Community Colleges 



Core Dimension 


Description 


Examples of Key Indicators 3 


Integration 


Curricular or programmatic linkages organized 
around common themes in order to construct 
shared, relevant teaching and learning 
experiences. 


Linked, interdisciplinary 
courses; contextualized 
curriculum; merged syllabi, 
integrated curricula; attached 
seminars; joint 
assignments and grading 


Active-learning 

pedagogy 


Use of pedagogy that promotes critical thinking 
through experiential, collaborative, and reflective 
learning. Purposeful classroom and co-curricular 
activities that relate course content to real-world 
issues/events. 


Problem- or project-based 
assignments; interactive 
classroom dialogue; small 
group or dyads; labs or field 
study 


Faculty 

engagement 


Opportunities for instructors to work together on 
linking activities and assignments across subjects 
and to share or develop effective pedagogy and 
community-building approaches. 


Co-teaching, team teaching; 
cross-course schedule 
planning; professional 
development to support 
teaching in learning 
communities 


Student 

engagement 


Opportunities for students to create meaningful 
peer networks that promote academic support and 
social bonding. Students are encouraged to play 
an active role and have an active voice in their 
shared learning. 


Cohorts; peer advisers/ 
mentors; study groups; 
informal social events and co- 
curricular activities 


Supplemental 
student support 
services 


Knowledge and utilization of campus resources 
that enhance the academic and social learning 
experiences of learning community students. 
Integration of these services with classroom 
activities. 


Counseling; tutoring; financial 
aid; job/career development; 
research/library resources; 
access to computers 
labs/laptops 


Institutional/ 

structural 

transformation 


Recognition of and institutional support for 
learning communities from the larger campus 
community. Ensuring that learning community 
objectives fit well with the overall institutional 
mission and objectives. 


Support/involvement from top 
administration; funding for 
sustainability efforts, 
enrollment outreach and 
marketing to students 



NOTE: 

“Appendix A contains a list of all potential indicators for each dimension. 



25 




first cohort and fall 2008 for the second cohort) and during the fourth semester following ran- 
dom assignment (spring 2009 for the first cohort and fall 2009 for the second cohort). 

Since each learning communities program is based on a unique configuration of courses 
and experiences, field protocols will be designed to capture both the core dimensions of learn- 
ing communities and the site-specific components at each college, as well as the standard expe- 
rience of students at each college. Field researchers will conduct a variety of activities while on- 
site, and each activity is designed to measure specific indicators. 

• Open-ended interviews will be conducted with college administrators and 
program staff associated both with learning communities courses and with 
the standard developmental courses. These interview sessions are designed to 
document the histories and current status of the learning communities pro- 
grams, any changes made to the original design of the programs and reasons 
for those changes, and the lessons learned during the implementation. Inter- 
view responses can also gauge the administrative knowledge and support for 
learning communities programs at the colleges, how the programs generally 
fit in with other college initiatives, an understanding of any large institutional 
issues or challenges the colleges face — or are likely to face — that may af- 
fect the long-term efforts to institutionalize learning communities on campus, 
and what steps learning communities administrators and staff have underta- 
ken, if any, to sustain their programs after the demonstration is over. 

• Faculty focus groups will be conducted with instructors both in and outside 
learning communities program. These focus groups will provide an opportu- 
nity to learn about the similarities and distinctions that exist between teaching 
in learning communities and standard developmental courses (for example, 
pedagogical practices, curricular structure, course scheduling), the nature of 
faculty and student interactions within both the treatment and the control 
classroom environment, and how faculty come to be recruited and trained for 
learning communities. 

• Student focus groups will be conducted with students enrolled both in 
learning communities programs and in stand-alone versions of the courses in 
the learning communities. Students who participate in these focus groups will 
be asked to volunteer their thoughts about what teaching techniques they 
think have best helped them leam course content, the nature of their interac- 
tions with faculty and other students in class, and their knowledge and use of 
campus resources (such as counseling, tutoring, and library services) to aid in 
their understanding of course content. 
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• Field researchers will conduct classroom observations in a small sample of 
learning communities. These observations will provide qualitative data on 
pedagogical approaches, interactions among students and between students 
and faculty, evidence of integration, and the physical settings of particular 
learning communities. 

Faculty Survey 

About 50 faculty members from each college will be asked to complete a questionnaire. 
These will include as many faculty members teaching in the learning communities programs as 
possible as well as a sample of faculty members who teach comparable, stand-alone courses. In 
order to measure the qualifications, attitudes, and pedagogical practices of instructors, the sur- 
vey will include items such as instructors’ confidence in students’ ability to learn content, types 
and levels of interactions with students inside and outside the classroom, professional develop- 
ment experiences, collaborative endeavors with other faculty and staff, use of other campus re- 
sources to enhance teaching and learning, and perceptions of student performance and effort in 
their courses. 

In addition to providing insight into promising practices within the classrooms of learn- 
ing communities, the faculty survey will serve as an important source of information about the 
differences between faculty who teach in the learning communities and those who do not. The 
demonstration is not designed to control for “teacher effects,” which would have necessitated 
random assignment of faculty to learning communities and to stand-alone developmental 
courses. But the implementation study — primarily through the faculty survey — will docu- 
ment and report measurable differences in terms of their background, experience, full-time or 
part-time status, and other variables that may be associated with student outcomes. 

The faculty survey will be administered in fall 2008 to the first cohort of colleges and in 
spring 2009 to the second cohort. 

Review of Documents 

Various program documents that describe planning and ongoing activities and events 
related to the learning communities programs will be analyzed. These documents will include 
course syllabi or course descriptions (to look for evidence of alignment or integration across 
courses) and, if available, teacher logs of classroom interactions, coursework assignments and 
activities, descriptions of group project assignments, student writing samples, and student eval- 
uations of teachers. 
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The Time Line for the Implementation Study 

Table 4 summarizes the schedule of data collection activities planned for the implemen- 
tation study and shows how the implementation findings will be incorporated into the project’s 
reporting schedule. Site visits are scheduled to occur in the second and fourth semesters follow- 
ing random assignment. While the collection of some documents for analysis will take place 
during the scheduled implementation research site visits, other document collection will take 
place prior to or following the site visits. 



The Impact Study 

Given both the limited evidence of the benefit of learning communities for students 
who are academically underprepared and the pattern of low persistence in college among this 
group, more research is needed to discover whether learning communities will engage such stu- 
dents and result in better outcomes. The impact study seeks to answer the following questions: 52 

1. What are the effects of learning communities on student achievement, as 
measured by assessment test scores in English and math, credits earned in 
developmental and regular college courses, grades, and other outcomes? 

2. What are the effects of learning communities on students’ academic persis- 
tence in higher education? 

Methodology 

As discussed above, a random assignment design will be used to answer these ques- 
tions. Random assignment is considered the “gold standard” of analysis because tests of differ- 
ences between groups that are randomly assigned are less biased and more powerful than simi- 
lar tests between nonequivalent groups, allowing such differences to be interpreted as causal. As 
a result, greater reliance may be placed on the magnitude and inference of impact estimates de- 
rived from this methodology. 

Data and Outcomes of Interest 

To measure the effects of learning communities on the acquisition of basic skills, course 
completion, persistence, and other outcomes, the study will rely principally on student tran- 



52 Separate analyses of state-level data by NCPR partners will inform how the results of the impact study 
may generalize to other institutions, by comparing students, schools, and colleges in this demonstration with 
other students in selected states. This analysis will also draw conclusions about the efficacy of other types of 
remediation — relative to learning communities — in improving the success of academically underprepared 
students. 
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The Learning Communities Demonstration 
Table 4 



Time Line for the Implementation Study 





Fall 

2007 


Spring 

2008 


Fall 

2008 


Spring 

2009 


Fall 

2009 


Spring 

2010 


Fall 

2010 


Spring 

2011 


Fall 

2011 


First cohort 
(KCC, QCC, 
HCC) 


Random 

assignment 

begins 


Round 1 
research 
visit 


Faculty 

survey 


Round 2 
research 
visit 
















Document collection -> 










Second cohort 
(Merced, 
HCCS, CCBC) 




Random 

assignment 

begins 


Round 1 
research 
visit 


Faculty 

survey 


Round 2 
research 
visit 
















Document Collection -> 








Reports 










Interim 

report 






Final 

report 


How-to 

guide 





scripts provided by the participating colleges. These data will be collected for up to two years 
after random assignment for everyone in the sample. When study sites are part of larger systems 
(for example, QCC and KCC in the City University of New York), transcripts for sample mem- 
bers who move to other institutions within the system may be gathered from central offices. 
Building on MDRC’s experience with the Opening Doors demonstration, the following out- 
come measures are typically available: registration for any courses, number of courses at- 
tempted (regular and developmental), number of courses passed (regular and developmental), 
withdrawal from one or more courses, status of English and/or math placement tests, total 
semesters enrolled, total credits earned, and grade point average (GPA). In addition, course- 
level information — such as course title and possible credits — major, and transfer to other 
postsecondary institutions may be collected. If available, data from a uniform test of basic read- 
ing, writing, and math skills both at baseline and at a follow-up point (most likely one or two 
semesters after random assignment) for all program and control group members may be col- 
lected. Having two data points will allow the study to measure the effectiveness of learning 
communities on increasing pass rates for these important assessments. 

As a secondary measure of persistence in postsecondary education, data from the Nation- 
al Student Clearinghouse will also be employed. This database, which captures information from 
91 percent of college students nationwide, will be used to create measures for students in the sam- 
ple who transfer to other colleges or universities. For example, it will allow the tracking of trans- 
fers to four-year universities, which is widely regarded as one measure of student success. 53 

Size of Sample Needed 

How many students are needed to detect policy-relevant effects on achievement, persis- 
tence, and retention? Overall, the experimental research sample at each site will comprise low- 
er-division community college students whom the colleges identify as in need of developmental 
English or math. 54 Table 5 lists additional eligibility criteria, by site. 

While each site is expected to enroll a minimum of 1,000 students into the research 
sample over a two-year period, some sites may exceed or fall short of this goal. Based on data 
from the Opening Doors demonstration, Table 6 shows the minimum detectable effect size 
(MDES) that can be detected with a given sample size and various ratios of treatment group to 
control group members (random assignment ratios). An MDES is a simple way to express the 
statistical precision of an impact study design. Intuitively, it is the smallest program impact that 



53 Rouse (1995) and Alba and Lavin (1981), for example, find that community colleges divert some stu- 
dents from attending four-year universities and that such students, in turn, complete fewer years of education. 

54 The exception to this is KCC, which is implementing career-focused, second-semester learning com- 
munities for students who have passed all remediation requirements. 
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The Learning Communities Demonstration 
Table 5 



Determinants of Eligibility for the Learning Communities Demonstration, by College 



Sample 

Selection 

Criteria 


KCC 


QCC 


HCC 


Merced 


HCCS 


CCBS 


Age 


1 8 and over 


18 and over 


1 8 and over 


1 8 and over 


18 and over 


1 8 and over 


Collegiate 

level 


Continuing or 

transfer 

students 


First-year students meeting remediation criteria; transfer 
students with fewer than 15 credits and returning students if 
failed lowest-level remedial math previously 


First-year 

students 


First-year 

and 

continuing 

students 


First-year 

students 


First-year 

students, 

continuing 

students, and 

transfer 

students 


Remediation 

level 


None 


Math (lowest level only) 


Reading (all 
levels) 


English (all 
levels) 


Math (lowest 
level only) 


English or 
reading 
(highest levels 
only) 


Subgroups 
of interest 


Gender; Allied 
Health majors; 
Nursing 
majors 


Gender; students requiring 2 or more remedial classes 


Gender; 

students 

requiring 2 or 

more 

remedial 

classes 


Gender; 
students 
requiring 2 
or more 
remedial 
classes 


Gender; 
students 
requiring 2 or 
more remedial 
classes 


Gender; 
students 
requiring 2 or 
more remedial 
classes 





The Learning Communities Demonstration 
Table 6 



Minimum Detectable Effect Size (MDES) (%), by 
Various Total Sample Sizes and Ratios of 
Treatment Group Members to Control Group Members 



Random Assignment Ratio 



Total Number of Students 


50:50 


60:40 


70:30 


80:20 


90:10 


400 


0.25 


0.25 


0.27 


0.31 


0.42 


600 


0.20 


0.21 


0.22 


0.25 


0.34 


800 


0.18 


0.18 


0.19 


0.22 


0.29 


1,000 


0.16 


0.16 


0.17 


0.20 


0.26 


1,200 


0.14 


0.15 


0.16 


0.18 


0.24 


1,400 


0.13 


0.14 


0.15 


0.17 


0.22 


1,600 


0.12 


0.13 


0.14 


0.16 


0.21 



NOTE: 

Calculations assume a 10 percent significant level, 80 percent power, and two-tailed test. See Appendix A 
for the exact formula employed in the table. 



can be measured with confidence, given random sampling and estimation error. 55 This metric, 
which is used widely for measuring the impacts of educational programs, is defined in terms of 
the underlying population’s standard deviation of student achievement. For example, an MDES 
of 0.25 indicates that the study can reliably detect (80 percent of the time) a program-induced 
increase in student achievement that is equal to or greater than 0.25 standard deviation of the 
existing student distribution. Impacts that are more precise than this (that is, smaller than this) 
have a smaller likelihood or power of being detected by the study. 



55 Bloom (1995). An MDES is the smallest true program effect size that would have an 80 percent chance 
of being detected (have 80 percent power) using a two-tailed hypothesis test at the 10 percent level of statistical 
significance. 
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Table 6 presents the MDES for different total sample sizes and random assignment ra- 
tios, assuming a significance level of 10 percent, 80 percent power, and a two-tailed test. 56 The 
fourth row of the table shows that, under the proposed total sample size of 1,000 students equal- 
ly divided between treatment group and control group members, each site will be able to detect 
effect sizes as small as 0.16. If the random assignment ratio is changed to 60 percent treatment 
group members and 40 percent control group members (the second column), the study is still 
able to detect impacts of this magnitude. As the ratio of treatment group members to control 
group members becomes extremely unbalanced — as in the 90: 10 split in the rightmost column 
— the MDES increases noticeably to 0.26 standard deviation. All sites in the demonstration are 
either equally balanced between treatment group members and control group members or have 
a 60:40 split. The latter change in random assignment ratio was implemented to accommodate 
the circumstances at specific sites. 57 

Although judgments about whether a specific effect size is large or small are somewhat 
arbitrary, several useful guidelines exist. Many researchers use a rule of thumb proposed by Co- 
hen, who suggests that effect sizes of roughly 0.20 be considered small, 0.50 be considered 
moderate, and 0.80 be considered large. 58 Lipsey provides empirical support for this typology 
based on the distribution of 102 mean effect sizes obtained from 186 meta-analyses of treatment 
effectiveness studies, most of which come from education research. The bottom third of his ob- 
served distribution (small effects) ranges from zero to 0.32; the middle third (moderate effects) 
ranges from 0.33 to 0.55; and the top third (large effects) ranges from 0.56 to 1.26. 59 

Another benchmark for effect size estimates are those from high school interventions 
that average 0.25 standard deviation and those of small learning groups in the Science, Tech- 
nology, Engineering, and Mathematics (STEM) fields in community colleges that average 0.21 
standard deviation. 60 Given these benchmarks, the study sample size is able to detect relatively 
small effects. 

Analytic Strategy 

The primary analytic method to determine program effects will be comparing average 
outcomes for the program and control group members, using standard statistical tests, such as 
t-tests. This will generate estimates of the impact of offering access to the learning communities 



56 A two-tailed test is used because the central policy issue to be addressed by the demonstration is whether 
the program has any effect on student performance. 

57 At several sites, the random assignment ratio was changed to 60:40 in order to prevent the canceling of 
learning communities for which faculty had already been assigned. 

58 Cohen (1988). 

59 Lipsey (1990). 

60 Hill, Bloom, Black, and Lipsey (2007); Springer, Stanne, and Donovan (1998). 
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program (the intention-to-treat). A linear model will be used to estimate the impact of learning 
communities on outcomes: 



y tj =r 00 +ro l r i+ r ij’ 

where y tj represents an outcome (such as the number of credits attempted) for student i in 
learning community j, and T t is an indicator variable equal to 1 for students randomly assigned 
to the treatment group. To account for the clustering of half the sample into groups of learning 
communities, heteroscedasticity-consistent White standard errors will be employed. 61 

The coefficient of interest is y 0] , as it represents the effect of assignment to a learning 
community on the outcome of interest. Because of the random assignment process, ordinary 
least squares (OLS) estimation of y m will provide an unbiased estimate of the “intention-to- 
treat” effect, and it is not necessary to control for other student characteristics. Note that the in- 
tent-to-treat coefficient estimates the effect of assigning a student to the treatment group on the 
outcome in question. While it estimates the gains that a policymaker can realistically expect to 
observe from implementing the program (since one cannot fully control for whether students 
actually participate), it does not necessarily represent the effect of the reforms for those who 
actually use them. 

Subgroups and Restricted Pooling Across Sites 62 

Two subgroups are of policy interest and will be analyzed separately for each site: 

• Gender. Compared with females, males are both less likely to enter postse- 
condary education and at a higher risk of dropping out once enrolled. 63 This 
problem is particularly acute among men of color. 64 Moreover, results from 
an earlier MDRC evaluation of learning communities at KCC showed that 



6l Greene (1997). Many educational researchers employ hierarchical linear models to account explicitly for 
the nesting often evident in educational research. In this design, only a portion of the sample is nested at the 
level of the analysis (that is, at the student level). While hierarchical linear models have been proposed for this 
situation (see Bauer, Sterba, and Hallfors, forthcoming), these methods appear to be very sensitive to noncom- 
pliance (authors’ analysis). Having analyzed previous work utilizing both methods of accounting for clustering, 
the authors find the White-corrected standard error approach to be unbiased. 

62 While the study is sufficiently powered to detect relatively small overall impacts, it can detect only rela- 
tively large subgroup impacts. This is indicated in Table 4.4, which show that, with smaller total sample sizes 
(similar to subgroup sample sizes), the MDES that can be detected is much larger. 

63 Tinto (1993). 

64 Cameron and Heckman (2001). 
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the magnitude of several of the impacts was larger for males than for fe- 
males . 65 

• Need for remediation. Studies show that students who must take two or 
more remedial courses (for example, one in English and one in math) are at a 
higher risk of dropping out and that the dropout rate increases with the num- 
ber of courses required . 66 Results from an earlier MDRC evaluation of learn- 
ing communities at KCC showed that students who needed remediation in 
two areas (math and English) benefited more from learning communities 
than did students who needed remediation in just one area . 67 

While the study has a sufficiently large sample at each site to analyze each site separate- 
ly, data on the gender and remedial subgroups or on specific outcomes may also be pooled 
across certain sites in cases where more precision or more power is needed to calculate results. 
Pooling will be based on similarities in sites’ learning communities models; for instance, HCC, 
HCCS, and Merced link developmental courses with a student success course, and QCC and 
HCCS link developmental math with other courses. 



The Cost Study 

Any findings of positive impacts of learning communities on student outcomes should 
be weighed against the cost of conducting “business as usual” for developmental and other stu- 
dents. Learning communities are likely to cost more than regular developmental classes because 
of such enhancements as additional faculty time, professional development costs, and, often, 
smaller class sizes. But if learning communities cost significantly more than stand-alone courses 
and produce little or only modest impacts, scarce resources may better be spent funding less 
costly programs with equal or larger benefits. This cost study will be conducted in at least two 
sites, purposely selected to represent contrasting learning communities models. 

The cost study will include estimates of the institutional costs of both the overall pro- 
gram and selected components of the program. A standard protocol and simple metrics, such as 
cost per student enrolled, will ensure comparability across institutions. 

Since learning communities are designed to increase student persistence, if the program 
is successful, community college costs will likely increase as a result of students’ attending the 



65 Scrivener et al. (2008). The KCC evaluation was not sufficiently powered to determine whether the ef- 
fects for males were significantly different than those for females; pooling across sites in the current demon- 
stration will create a large enough sample to determine statistical significance. 

66 Adelman (2004); Attewell, Lavin, Domina, and Levey (2006); Adelman (1998). 

67 Scrivener et al. (2008). 
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community college for a longer period of time. However, it is important to distinguish the costs 
of operating the learning communities from the costs of additional community college atten- 
dance as a result of the intervention. This can be accomplished by focusing on the period of 
time when students are expected to receive the intervention and measuring all relevant costs for 
both the program and the control group. The difference in costs during this period will be attri- 
buted to the net cost of providing learning communities. Any costs associated with student per- 
sistence will not be considered a learning community cost. 

Component Costs of Learning Communities 

Separate per-person cost estimates will be calculated for each relevant component af- 
fected by the learning community models. Possible components include the following: 

• Classes. The costs of class instruction provided to program group members 
might be compared with the costs of similar classes provided to control 
group members. Examples of such classes include basic writing, reading, and 
social sciences. These will be linked classes for the program group but stand- 
alone classes for the control group. Differences in costs between the two 
groups might be attributable to differences in class size, the number of facul- 
ty, and hours of instruction. 

• Group activities. Smaller groups within a learning community might be 
formed around particular interests or one-time events, such as field trips, so- 
cial activities, and seminars. 

• One-on-one assistance. Learning community services such as tutoring, 
counseling, and case management would be calculated separately. 

• Training and development. This component would include any extra train- 
ing provided to staff to implement the learning communities model. 

• Program coordination. Costs associated with having a learning community 
coordinator would be captured in this component. The costs associated with 
implementing the evaluation (for example, assisting with the random as- 
signment process) will be subtracted, since these are research costs and not 
the costs of implementing the program. 

Methodology 

To estimate the per-person component costs above, the cost study will begin by first es- 
timating unit costs: the costs of providing component services to one person over a specific time 
period. These can be determined by first calculating the total cost of providing a given program 
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component over the time period (whether a class session, a semester, a week, a month, or a 
year) and then dividing by the number of individuals who participated in the activity or received 
the service over the corresponding time period. Thus, data are needed on both the total cost and 
the number of participants during the time period. 

The unit cost for a component will then be multiplied by the duration of participation in 
or receipt of the component. For example, if a unit cost of a particular class is calculated at $150 
per semester credit hour, $150 would be multiplied by the average number of semester credit 
hours that the program group members received. Different unit cost estimates will be calculated 
for the program group components and the control group components. 

The gross cost per program or control group member is simply the aggregate compo- 
nent costs per person calculated above. The difference between the gross cost per program 
group member and the gross cost per control group member yields the net cost of the learning 
community program. 

Cost data will be collected beginning in 2009 and will be analyzed separately by site, 
including estimates for unit costs per component, participation estimates, gross cost per program 
and control group member, and net cost. Any variation in costs across the sites will be explained 
using implementation data. Differences might be attributed to differences in the mix of activities 
provided, in unit cost estimates, and/or in participation levels. 
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Appendix A 



Core Dimensions of Learning Communities 
and Their Indicators 




Appendix Table A.l 

Data Collection Sources 

Interviews I Focus Groups j 



List of Core Dimensions 

Integration 

Integrated or linked curricula between two or 
more courses 
Merged syllabi 

Assignments built around common themes/topics 
Use multiple perspectives to explain content and 
examine issues 

Identification of connections and conflicts among 
diverse disciplines 

Attached seminars, special topics lectures, or 
research/field study sessions linked to courses 
Joint assignments, projects, and grading practices 
across courses or disciplines 
Blocked or aligned course schedule 

Active-Learning Pedagogy 

Project- and problem-based learning opportunities 
in the classroom 

Interactive dialogue or discussion regarding 
content between students and faculty 
members 

Small group or dyads that promote cooperative 
learning and critical thinking 
Reflective or responsive writing opportunities 
Interactive labs and field study that enhance 
content learning 

Classroom practices and discussion around 
diversity and equity issues 
Instructors actively reach out to students 
experiencing difficulty during class 



President Faculty Dean(s) 



Student 

Services 



Focus Groups 
tculty Students 


Faculty 

Survey 


Observations 


Documents/ 

Records 

Review 


X 


X 


X 


X 




X 


X 


X 




X 


X 


X 


X 


X 


X 


X 




X 


X 




X 


X 


X 


X 




X 


X 


X 


X 




X 


X 


X 




X 


X 








X 


X 


X 


X 


X 




X 


X 


X 


X 




X 


X 


X 


X 




X 


X 


X 


X 


X 


X 


X 


X 


X 




X 


X 


X 


X 




X 




X 







(continued) 



NOTE: This matrix includes all potential indicators for each dimension and the data sources where they can be found. It was created as a tool to guide protocol 
development for the implementation study. The final protocols for each of these data sources will include instruments to measure selected indicators. 





Appendix 



Interviews 



List of Core Dimensions President Faculty Dean(s) 

Faculty Engagement 

Collaborative course planning and teaching 

between faculty in linked courses X 

Regular, ongoing meetings and communication 

between faculty members during term X 

Faculty work together to develop theme-based 

assignments or projects X 

Team teaching or co-teaching across courses or 

disciplines X 

Opportunities for professional development to 
learn and hone learning community 

pedagogical practices X 

4^ Compensation or incentives for faculty to 
to participate in learning communities 

Faculty promote a sense of community, of 
belonging, and of shared enterprise with 
students and other faculty in the learning 

community classroom X 

Student Engagement 

Cohort of students enrolled in paired/multilinked 

and/or sequential courses X 

Creation of faculty-mandated or informal 

student dyads or study groups X 

Diverse student groupings across race, culture, 

gender, and so on. X 

Co-curricular social activities or events 
connected to courses (such as field trips, 
service learning or community-based 

projects, meal gatherings) X 

Informal social interactions or bonding 
between learning community students 

outside class X 

Learning community student perception of 
more intimate or meaningful relationships 
with cohorts than with other students 
Learning community students develop confidence 
as independent learners 



X 






Appendix Table A.l (continued) 





Data Collection Sources 




Interviews 


Focus Groups 


























Documents/ 










Student 






Faculty 




Records 


List of Core Dimensions 


President 


Faculty 


Dean(s) 


Services 


Faculty 


Students 


Survey 


Observations 


Review 


Supplemental Student Support Services 

Increased knowledge and use of various 




















academic support services by learning 
community students (such as counselors, 
tutors, finacial aid, library resources.) 




X 




X 


X 


X 


X 






Student service staff contribute to syllabus or 




















curriculum development 




X 




X 


X 




X 




X 


Student service representatives make 




















presentations in classrooms regarding 
campus resources 




X 




X 






X 




X 


Student services staff and administrators meet 




















with faculty regarding student academic or 
social needs 




X 




X 


X 




X 






Faculty actively refer students to campus 




















resources 




X 




X 


X 




X 






Perception of collaborative relationships 




















between instruction and student services 




X 


X 


X 


X 




X 






Institutional/Structural T ransfor mation 

Increased awareness of learning communities 




















by administration and larger campus 
community 


X 


X 


X 


X 


X 




X 






Perception of learning communities as an 




















institutional priority 


X 


X 


X 


X 


X 




X 






History of learning communities at college 


X 


X 


X 


X 












"Fit" between learning communities 




















goals/outcomes and mission of college 


X 


X 


X 


X 


X 




X 






Sustainable funding for learning community 




















programs growth and development 


X 


X 


X 




X 




X 






Regular communication between upper-level 




















administrators and frontline learning 
community staff 


X 


X 


X 


X 


X 




X 






Targeted, clearly defined learning outcomes 
Prevalence of LC-inspired pedagogical practices in 




X 






X 


X 


X 






non-learning community settings 


X 


X 


X 


X 


X 




X 







SOURCES: 

Smith, MacGregor, Matthews, and Gabelnick (2004); Engstrom and Tinto (2007); MDRC (2005); and Oertel (2001). 
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